Skip to content

Fixes for single event sequences#47

Open
anantma wants to merge 1 commit intoMogeng:masterfrom
anantma:single-event-sequence
Open

Fixes for single event sequences#47
anantma wants to merge 1 commit intoMogeng:masterfrom
anantma:single-event-sequence

Conversation

@anantma
Copy link

@anantma anantma commented Jan 14, 2026

Gracefully handled single event sequences during training.
The library's linear models are patched to handle empty data arrays from single-event sequences, preventing errors during the transition calculation step. This aligns the implementation with HMM theory, which permits sequences of any length, including single-event sequences that have no transitions.


This change is Reviewable

@Mogeng
Copy link
Owner

Mogeng commented Jan 14, 2026

Hi @anantma, thanks for your interests! I would prefer the filtering of 0 size data happen during 1) application time (before you use the package, remove unusable sequences first); Or 2) do it during the set_data stage (remove all df that have df.shape[0] == 0). Let me know what you think.

@anantma
Copy link
Author

anantma commented Jan 23, 2026

Hi @Mogeng , thank you for the response.

I encountered training errors when any sequence in the data has length == 1. Specifically, during transition model estimation (no transitions with single event sequences). Here is a minimal reproducible example -

import pandas as pd
from IOHMM import UnSupervisedIOHMM, OLS, CrossEntropyMNL

# Create a single length sequence
df_single = pd.DataFrame({
    'rt': [5.0],  # response
    'feature': [1.0]  # covariate
})
  
# Set up IOHMM model
model = UnSupervisedIOHMM(num_states=1, max_EM_iter=10, EM_tol=1e-4)
model.set_models(
    model_initial=CrossEntropyMNL(solver='lbfgs'),
    model_transition=CrossEntropyMNL(solver='lbfgs'),
    model_emissions=[OLS()]
)
model.set_inputs(
    covariates_initial=[],
    covariates_transition=['feature'],
    covariates_emissions=[['feature']]
)
model.set_outputs([['rt']])

# Set data and train
model.set_data([df_single])
model.train()

The relevant error trace -

  File "/root/IOHMM/test.py", line 26, in <module>
    model.train()
  File "/root/IOHMM/IOHMM/IOHMM.py", line 403, in train
    self.M_step()
  File "/root/IOHMM/IOHMM/IOHMM.py", line 383, in M_step
    self.model_transition[st].fit(X, Y)
  File "/root/IOHMM/IOHMM/linear_models.py", line 905, in fit
    X, sample_weight = self._transform_X_sample_weight(X, sample_weight=sample_weight)
  File "/root/IOHMM/IOHMM/linear_models.py", line 180, in _transform_X_sample_weight
    X = self._transform_X(X)
  File "/root/IOHMM/IOHMM/linear_models.py", line 149, in _transform_X
    X = add_constant(X, has_constant='add')
  File "/root/IOHMM/.venv/lib/python3.10/site-packages/statsmodels/tools/tools.py", line 195, in add_constant
    is_nonzero_const = np.ptp(x, axis=0) == 0
  File "/root/IOHMM/.venv/lib/python3.10/site-packages/numpy/core/fromnumeric.py", line 2684, in ptp
    return _methods._ptp(a, axis=axis, out=out, **kwargs)
  File "/root/IOHMM/.venv/lib/python3.10/site-packages/numpy/core/_methods.py", line 220, in _ptp
    umr_maximum(a, axis, None, out, keepdims),
ValueError: zero-size array to reduction operation maximum which has no identity

I believe it would make sense to keep these single length sequences as they would contribute to initial/emission model estimation, i.e. they should be treated as valid samples - hence the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments